Connections to Gaussian processes

Hover over the links for details on some connections. Dots indicate ‘special cases of’. Undirected connections suggest equivalence (≡ on hover) or a more complex relationship (usually for more general/more complex models). Note this is for a quick reference, not an exhaustive one.

Matern class

Note that covariance functions are written identically in slightly different ways in different sources, and this is just one representation. See the Rasmussen and Williams link for details.

\[h^2\frac{2^{1-\nu}}{\Gamma(\nu)}(2\sqrt{\nu}\frac{|x_i-x_j|}{\lambda})\mathcal{B}_\nu(2\sqrt{\nu}\frac{|x_i-x_j|}{\lambda})\]

\(\lambda\) = horizontal/input length-scale

\(h\) = vertical/output length-scale

\(\nu\) = controls differentiability

\(\Gamma\) = Gamma function

\(\mathcal{B}\) = modified Bessel function of the second kind

Labels

GP: Gaussian process

Matern: Matern covariance structure

Exp: exponential covariance structure \(h^2\exp(-\frac{|x_i-x_j|}{\lambda})\)

SqExp: squared exponential covariance structure \(h^2\exp[-(\frac{|x_i-x_j|}{\lambda})^2]\)

RQ: rational quadratic covariance structure \(h^2(1 + \frac{|x_i-x_j|^2}{\alpha\lambda^2})^{-\alpha}\)

Other: other covariance functions

OU: Ornstein-Uhlenbeck process

GAM: generalized additive models

Splines: piecewise polynomial, regression splines

SVM: support vector machines

NN: neural networks

RKHS: reproducing kernel hilbert space

References